Various pointing devices have been used in computer systems to input the relative movement data. One most often used pointing device is computer mouse. A computer mouse can be moved by a user on a flat surface. A cursor displayed on the screen of a computer display follows the movement of the mouse and points on various items on the screen. While many people prefer using a mouse to other pointing devices, some people may suffer from pains in their fingers, hands, wrists, arms and shoulders after using a mouse for a long period of time. Alternative pointing devices such as trackballs, touchpads and joysticks can be used to reduce or avoid the use of mice for those people who suffer from the mouse-related pain.
The use of personal computers has been increased rapidly in recent years. Many people use computers at work for several hours per working day. At home, people use computers to access internet, read and write emails and play computer games. The new trend in the computer industry is to combine a computer with audio and video equipment and sources to provide a multimedia home entertainment system at home. Users of such a computer system can sit on their sofa and control the various functionalities, such as selecting TV channels, playing and recording video, playing and recording music, showing digital photos and accessing internet, by using a hand-held remote control. One desirable functionality of such a remote control is the functionality of controlling cursor movement. This means a pointing device should be designed and implemented on the remote control. At present, trackball, joystick and miniature gyroscope have been used to implement the pointing functionality on the remote controls for multimedia home entertainment systems. A pen tablet can be used as both a pointing device and a handwriting device. When a user is pointing with a cursor on the screen of a computer display, he/she pays attention to the eye-hand coordination. When a user uses handwriting to input letters or numbers into a computer, he/she pays more attention to the movements of the pen rather than to the eye-hand coordination. Handwriting input is also provided on many hand-held personal computers. A user can use a stylus to write letters and numbers on the screen of a hand-held personal computer. A handwriting recognition software is executed on the computer to recognize letters and numbers.
There is thus a need for a portable free-space data input device. However, in order to use such a device it is necessary to provide data to be processed from the device together with the possibility to track the position of the device.
In addition to the alternative pointing devices mentioned above, there is another market-available pointing device that uses a gyroscope to measure the angular changes of the user's hand. The designs of such a device are described in U.S. Pat. Nos. 5,440,326, 5,698,784, 5,825,350 and 5,898,421. The pointing device can be held in a user's hand and moved in free space. This provides an alternative is means for the user to reduce the use of the ordinary mouse and is recommended for the people who suffer from the mouse-related pain.
One problem to be solved in the design of a hand-held pointing device is how to let the user to use more natural hand and wrist movements to move the device in free space. The pointing device based on a gyroscope allows the user to move the device in free space. However, due to the use of a gyroscope as an angle detector, the user tends to use large angular wrist movements to make large or quick movements of the cursor on the screen. These large angular wrist movements deviate very much from the neutral position of the user's wrist. Therefore, the user may feel tired in his/her wrist if many large wrist movements are made for a long period of time. Since a gyroscope detects the angle movement of the user's hand, the pointing device cannot be used naturally as a free-space handwriting device, which requires the detection of the position of the user's hand in free space.
Another market-available pointing device, called SmartNav3™ marketed by NaturalPoint Inc., that can be used by the people who suffer from the mouse-related pain uses a two-dimensional imaging sensor to detect the movement of the head or hand of a user. The user can move his/her head or hand in free space to control the movement of the cursor. An infrared light source and an infrared camera are placed in front of the user. The user should put a small piece of reflective material on his/her head, glasses or cap or wear a ring with a small piece of reflective material. When the user moves his/her head or hand within the viewing angle of the camera, a sequence of two-dimensional images is captured by the camera. Each captured image is pre-processed by a special digital circuit to extract the high intensity area, which corresponds to the small piece of the reflective material. The pre-processed image is then sent to the personal computer via USB (Universal Serial Bus) port for further processing. The software on the computer detects the position of the high intensity area in the pre-processed image and converts the position changes to cursor movements. It is also possible to use a hand-held infrared light source and let the infrared camera directly capture the image of the light source.
Large angular wrist movements can be avoided when a pointing device based on a two-dimensional imaging sensor is used. The typical movement of the user's hand in free space is different from that of moving a pointing device based on a gyroscope. Since the user can move the three-dimensional position of the light source quickly by using his/her arm, the user does not need to make large angular wrist movements. Therefore, a to pointing device based on a two-dimensional imaging sensor is better than a pointing device based on a gyroscope from an ergonomic point of view.
At present, the above-mentioned market-available pointing device uses a two-dimensional imaging sensor of a limited resolution (i.e., a limited number of pixels) and thus has a rather limited angle of view. The infrared reflector or the infrared light source must be moved within the angles of view of the camera. If a user uses his/her head to control the cursor movement, the narrow angle of view can be accepted. But, if a user uses his/her hand to control the cursor movement, he/she may feel inconvenient if the angle of view is too narrow. The viewing angle of a camera cannot be simply increased by using a wide-angle lens. Due to the limited resolution of the two-dimensional imaging sensor, larger hand movement is required when the angle of view of the camera is increased. With a larger viewing angle, the required hand movement is further increased when the infrared light source is moved to a position more distant from the camera. A user may feel inconvenient to make a very large hand movement. Therefore, it is desirable to increase the number of pixels of the two-dimensional imaging sensor of the camera. However, the increased cost of a larger two-dimensional imaging sensor and the requirement on the increased throughput of the data processing hardware may result in a more expensive product.
Another potential problem of the market-available pointing device based a two-dimensional imaging sensor is that if there are more than one infrared light sources or reflective objects in the field of view of the camera, the image processing algorithm of the device may fail to detect the position of the intended infrared light source or the piece of reflective material worn by the user. This means that ambient light and any disturbing reflective material in the field of view of the camera should be controlled so that this problem can be avoided. If a pointing device based on an optical position detector should be designed, the problem of disturbing ambient light should be solved.
If a pointing device based an optical position detector should be used as a handwriting device in free space, a button must be available on the portable part of the device such that it can be operated by the moving hand of a user to activate handwriting. The market-available available pointing device based on a two-dimensional imaging sensor does not have any button on the portable part for activating handwriting. Using a button on a keyboard or a foot-operable switch for activating handwriting is not convenient.
It would therefore be advantageous to allow the use of a portable free-space data input device that does not require large angular wrist movements, while at the same time using an optical position detection technique that can be kept simple in its design.
There does exist an old technology for optical position detection of among other things a pilots head using one-dimensional imaging sensors and slits to detect the movement of a single point of light source. However it has never been used in combination with inputting data and is therefore not really close at hand to consider for use with data input devices. Since no input of data is used from the device having its position determined, the technology further has to use a light source that is constantly turned on.
This technology is for instance described in U.S. Pat. No. 4,092,072, where a single horizontally-mounted one-dimensional imaging sensor is used together with two mutually inclined slits arranged in a V-shaped configuration to detect the position of a light source. The sheets of light passing through the two slits impinge on the one-dimensional imaging sensor and result in two pulses in the captured one-dimensional image. The vertical position of the light source can be determined by measuring the distance between the two pulses. The horizontal position of the light source can be computed from the positions of the pulses relative to one end of the imaging sensor. Instead of using two V-shaped slits, it is also possible to use a mask with a triangular aperture or a mask with a triangular light obstructing member with two mutually inclined edges. The light passing through a mask with a triangular aperture impinges on the plane with the one-dimensional imaging sensor and forms a triangular pattern on the plane. Since the triangular pattern moves horizontally and vertically on the plane corresponding to the horizontal and vertical movements of the light source, the intersection of the pattern and the one-dimensional imaging sensor results in a pulse of varying positions and widths in the captured one-dimensional image. The horizontal position of the light source can be computed from the positions of the edges of the pulse relative to one end of the imaging sensor. The vertical position of the light source can be determined by measuring the distance between the two edges of the pulse. If a mask with a triangular light obstructing member with two mutually inclined edges is used, a pattern of a shadow formed by the triangular light obstructing member and the border of the mask is projected on the plane with the one-dimensional imaging sensor. The intersection of the pattern and the one-dimensional imaging sensor results in two wide pulses of varying positions and widths in the captured one-dimensional image. The horizontal and vertical positions of the light source can be computed from the positions of the two edges, which correspond to the mutually inclined edges of the mask, in the one-dimensional image.
In U.S. Pat. No. 4,209,254, a single one-dimensional imaging sensor is used. The two slits are arranged in perpendicular directions. The light passing through the two slits form two sheets of light. One sheet of light intersects the one-dimensional imaging sensor perpendicularly. The other sheet of light is rotated by a set of prisms so that this sheet of light also intersects the one-dimensional imaging sensor perpendicularly. The vertical and horizontal positions of the light source can be determined by measuring the positions of the two pulses detected in the captured one-dimensional image. Cylindrical lenses can be used in place of the slits to focus the light onto the one-dimensional imaging sensor.
A sensing device described by U.S. Pat. No. 3,951,550 can detect the direction of a single point source with the aid of two slits perpendicular to each other in a mask otherwise opaque to the emitted radiation. These slits are traversed by flat beams or sheets of incident rays lying in mutually orthogonal planes whose orientation can be detected with the aid of two elongated radiation detectors respectively intersected by these beams.
Even though different light limiting means such as masks with two slits arranged in a V-shaped configuration, a triangular aperture, a triangular light obstructing member with two mutually inclined edges and two orthogonally-oriented slits with or without a set of light-beam-rotating prisms is used in the designs of optical position/direction detecting devices in U.S. Pat. Nos. 4,092,072, 4,209,254 and 3,951,550, one common feature of these designs is that they use only one-dimensional sensors to detect two-dimensional positions/directions of a single point of light source. This optical position detection technique is feasible because the light from the single point of light source is limited by the light limiting means such that a two-dimensional pattern is projected onto the plane(s) where the sensing elements of one-dimensional sensors are located. For example, in the case of a triangular aperture, the triangular pattern is two-dimensional and in the case of two orthogonally-oriented slits, one part of the pattern is a horizontally-oriented narrow rectangle and the other part is a vertically-oriented narrow rectangle. A pattern should be produced by the light limiting means in such a way that when the single point of light source moves in free space, its position changes in at least two dimensions must result in two-dimensional changes of the pattern as well. One or two one-dimensional imaging sensors can be used to capture the two-dimensional pattern changes on the plane(s). Whether one or two one-dimensional imaging sensors should be used depends on the light limiting means and the produced pattern. The algorithms for two-dimensional position computation to be used depend on both the light limiting means and how the sensor(s) is arranged in relation to the light limiting means.